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Introduction 



To mark its twenty-fifth anniversary, the National Centre for Vocational Education Research 
(NCVER) brought together policy, industry and academic leaders to reflect on the role that research 
and statistics have played in the development of Australia’s vocational education and training (VET) 
sector. The forum Informing policy and practice in Australia’ s vocational education and training sector: Kef lections 
and futures was held at the Stamford Plaza Hotel, Adelaide, on 21 March 2007. 

After an official welcoming from the current Chairman of the NCVER Board, Peter Grant, a 
former NCVER Managing Director, Chris Robinson, now Head of the Department of Education 
and Children’s Services in South Australia, reflected on NCVER’s role in promoting quality 
research and statistics in the Australian VET sector. 

An important role that NCVER has played over its history has been to provide a factual and 
analytical basis to many of the important national debates, skills shortages being just one, according 
to Mr Robinson. A future challenge for NCVER will be to continue to provide research that is 
valuable and has impact. A way of achieving this might be through undertaking a range of research 
in an area, rather than focusing on a single project approach. Peter Kearns of Global Learning 
Services, the discussant for this session, highlighted the need for greater attention to regions and to 
the international context. Incorporating new and emerging issues such as sustainability into VET 
practice, and research more generally, will become increasingly important in the future. 

Berwyn Clayton from the Canberra Institute of Technology reflected on the development of a VET 
research capacity in Australia. In particular, she highlighted the need to raise the profile of the 
research effort and make connections to policy and practice. Professor Andrew Smith of Charles 
Sturt University responded by warning of a potential future shortage in research capacity in VET 
due to ‘greying’ of the existing research community. Steps need to be taken now to build researcher 
capacity in the sector, and NCVER has a major role to play in this. 

Mark Cully of NCVER and Brian Knight of the Australian Department of Education, Science and 
Training discussed the development and growth of a mature program of VET statistics. The present 
national statistical program needs to continue to evolve, and challenges outlined by the speakers 
include obtaining more information on total VET effort and developing meaningful activity and 
output measures for VET. The discussant, Michelle Bruniges, Chief Executive of the Department 
of Education and Training in the Australian Capital Territory, agreed that the collection of training 
statistics had come a long way, but that we need to do more to respond to local and community needs. 

Finally, Tom Karmel, current Managing Director of NCVER, closed the forum by anticipating 
NCVER’s future. According to Dr Karmel, NCVER must continue to feed into policy by 
understanding the VET sector and how it fits into our economy and society. There will always be a 
role for an organisation such as NCVER while there exists the need for information and research 
to support policy-makers, practitioners and the wider community alike. 

These proceedings will be relevant to those interested in the development and growth of a VET 
research and statistical capacity in Australia. 



Phil Loveder 

Manager, International and Consultancy Services, NCVER 
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Evolution of NCVER and its role 
in the vocational education 
and training sector 



Chris Robinson, 
Department of Education and 
Children’s Services, South Australia 



Vocational education and training (VET) began early in Australia’s history — with convict 
transportation. Early Australian apprenticeships lasted nine years and the contract of training at that 
time was more complex than that of today, and certainly contained more data items on it. For 
example, potential apprentices had to pledge that they attended church on Sundays and that they 
wouldn’t misbehave generally. Over the years the contract of training has gradually been reformed 
and requests for more personal information about an apprentice’s life have been deleted. 

VET institutes developed around the country in various ways, for example, through mechanics 
institutes and technical high schools. Interestingly, it seems that technical high schools will once 
again become part of the VET landscape. 

In the 1970s, probably one of the most important reports ever released in relation to vocational 
education in Australia was that of Myer Kangan, which ultimately led to the development of the 
modern technical and further education (TAFE) system and the associated systematic and coherent 
funding of that system. TAFE — the public VET system — has been extremely successful, although 
over the years it has attracted some criticism. VET systems around the world suffer by comparison 
with the public VET system in Australia. 

One of the important outcomes of the Kangan report was a recommendation to establish an 
Australian TAFE Technology Centre. Although this didn’t eventuate, a short time later a further 
enquiry (the Committee of Inquiry into Education and Training — known as the Williams 
Committee) went one step further and recommended the development of a TAFE centre for 
research and development. The first element in implementing that recommendation was the 
establishment in 1979 of a national TAFE clearinghouse for research information. The actual 
research establishment, the TAFE National Centre for Research and Development, began in 1981. 

In 1981, the centre’s first director, Graham Hermann, was appointed, with the centre’s first review 
in 1984 and the subsequent recommendation that it should continue. Bill Hall was appointed 
director in 1984 and remained director for the centre’s formative years. 

The Australian Journal of TAFE Research started ini 985. Also around this time the heads of the 
TAFE system in the various jurisdictions initiated the second review of the centre. The 
recommendation from that review was that the centre should continue to receive core grant 
funding for research. 

The first direct role for the centre in statistical collection began in 1991 when it assumed 
responsibility for the national TAFE statistical collection. The Kirby enquiry into labour market 
programs demonstrated the need for the collection of national TAFE statistics. Until that time the 
statistical collections in this area were neither consistent nor representative. 
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The first issue of the Australian Training Review was released in 1991. In 1992 the TAFE National 
Centre for Research and Development was renamed the National Centre for Vocational Education 
Research. The Australian National Training Authority (ANTA) was also born that year. 

In the 1990s a landmark report was produced by Rod McDonald and a number of his colleagues at 
the University of Technology, Sydney — Geoff Hayton, Andrew Gonczi and Paul Hager. This 
report, which I’ll comment on in more detail later, had a profound impact on what followed. 

NCVER became involved in the apprenticeship and traineeship data collection in 1993, and in 1994 
began to fund additional VET research through a separate subcommittee, the Australian National 
Training Authority Research Advisory Council (ANTARAC). And 1995 marks a particularly 
important moment in VET history. This is when ‘the audit of non-financial VET activity’ 
commenced, with the aim of establishing whether the national training data given to NCVER by the 
states and territories could be verified at individual registered training organisation level. Through 
this mechanism, governments could feel assured that their money had been spent appropriately. 

In 1996 NCVER assumed responsibility for the graduate and student satisfaction surveys and the 
employer use and views of the VET system surveys. In that year I was appointed as director, and 
later that year arrangements were made for NCVER to manage a consolidated national competitive 
research program relating to the VET sector (overseen by the National Research and Evaluation 
Committee). At the same time ANTA established three VET research centres to undertake more 
VET research, although these centres had already been conducting research in this area. These 
centres, along with the University of Tasmania, produced a great deal of VET research in particular 
fields over the following years and made a significant contribution to the research effort. 

In 1997 the VOCED research database (the successor to the TAFE Clearinghouse) became an 
international database and went online. The clearinghouse had been a paper-based system, with a 
three-monthly report of research abstracts posted to various stakeholders. Today’s VOCED is a 
far more sophisticated research tool and has become a comprehensive national and international 
database. 

The first national VET research strategy was also developed at that time and covered the period 
from 1997 to 2000. In 1999 the centre and its research underwent another review, undertaken by a 
former chair, Rae Taylor. The recommendations from this review were largely concerned with the 
development of processes to combine the research and policy cycles and maximisation of the 
comprehensive VET statistical resources that NCVER was making available to the sector. 

In 2000 the first issue of Insight, the NCVER publication with topical summaries of recent research 
and other centre activities was released. The second national strategy for VET was developed in 
2001 to cover the next few years. In the same year NCVER became a UNESCO 1 Centre of 
Excellence jointly with TAFE South Australia, and the VOCED database became UNESCO’s 
officially recognised VET research database. 

Tom Karmel was appointed in 2002 as the centre’s fourth director, with the third national VET 
research strategy being released in 2003. 

ANTA was abolished in 2005 and its functions taken over by the Department of Education, 

Science and Training. Despite the abolition of ANTA, the important national role for NCVER in 
research and statistics continued. 

New national research priorities were agreed between NCVER and the Commonwealth and 
states in 2006. At the same time another review was commissioned and was undertaken by Neil 
Johnston AO. NCVER received overwhelming support from the vocational education and 
training system and stakeholders, confirming the increasing importance of all the different 
products and services produced by the centre. 



1 UNESCO — United Nations Educational, Scientific and Cultural Organization. 
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VET research and NCVER 



Earlier I mentioned an important research report by Rod McDonald and colleagues published in 
the 1990s. In retrospect, I consider that No small change was a pivotal document in terms of 
examining research and the researcher’s role in policy work. The report noted that at the time 
Australia had a fairly low level of VET research and development and most of what was done was 
being conducted by the centre, with a small amount undertaken by the Commonwealth 
Government and an even smaller amount being undertaken by the state VET agencies. In terms of 
actual volume, very little VET research was being conducted at the numerous research centres that 
did exist in universities. These were more focused on other education sectors, as was the Australian 
Council for Educational Research, which is now well over 50 years of age. 

The No small change report hinted that the VET sector made little use of formal evaluations. At that 
time it was certainly true that very little evaluation work was being undertaken, and perhaps even 
today this holds true. Certainly at that time VET research was having litde influence on VET 
policy-making. In fact, most of the VET research being done at that time was focused upon various 
impacts on practice, in other words, on the supply side of the equation. The report recommended a 
national VET research strategy. That led to the production of various pieces of research that 
examined the impact of VET research on policy. Those undertaking this work included Bill Hall, 
Perce Butterworth, Gregor Ramsay and Ken Wiltshire. Gerald Burke examined the impact of 
statistical and quantitative work on the sector. Chris Selby Smith, John Foyster and I also 
contributed to the debate at the time. As it turned out, all of this debate over the years 1993 to 1996 
became the lead-up to the first national VET strategy. 

Chris Selby Smith’s work at this time was a significant overview in research, arriving at a few 
significant conclusions on the impact of research on VET decision-making. Researchers at that 
time were highlighting the lack of an evidence base in much of the policy-making work. Policy- 
makers were focused at the macro level, while much of the research at that time was focused very 
narrowly on particular issues. As highlighted earlier, the policy and research cycles were not well 
linked at all, and VET research was not greatly influencing VET decision-making. 

As also noted above, all of this research work led to this first national VET strategy, which was 
really very important. It established some areas of investigation to focus on and to prioritise. As far 
as I know, it was the first national VET research strategy developed in any country and it was 
specifically aimed at research having a greater role in and impact on policy, and indeed on practice. 

In that first round, over 75 projects were undertaken or commissioned through the strategy. 

Research into the economic and social implications of VET was extremely rare before this strategy, 
so the priority became to increase significant research in these areas, with the aim of relating the 
research to policy development. However, research also continued to look at other issues of 
importance to the sector, such as the transition of young people from school to work, the quality of 
VET provision, and ways to increase the effectiveness of practice in the VET sector, all of which 
were also very significant priorities in the strategy. 

One of the first NCVER exercises under the new strategy was undertaking a scoping of the 
research in many areas and bringing the results together as a book of readings. At that time issues 
were emerging about training markets, VET in Schools was starting to take off and flexible delivery 
of training was another important area. Each of the chapters in this book reviewed the research 
work to date in Australia in these and other important areas, resulting in an extremely important 
scoping exercise — a synthesis of all the VET research over the previous decade. It was a very 
useful document at the time in gaining an understanding of the breadth and scope of previous 
research work. 

Also in 1997 NCVER decided to become involved in the substantial and heated debates about the 
market for VET. This led to the publication of a book of readings, The market for l TIT, which 
captured important national research on the topic. This was the time when the user choice policy was 
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being implemented. This move generated diametrically opposing views and I considered that it might 
be an interesting exercise to inform people of the various perspectives and opposing positions on 
this issue. In a conference organised by NCVER we were able to involve a large number of people 
who hadn’t been so directly part of the VET research community up until that time. This was 
largely because VET research had focused more on practice rather than on policies. People like 
Vince FitzGerald and Bruce Chapman spoke in terms of market models, while Kaye Schofield, 
when discussing the future of TAFE, referred to ‘radical surgery or palliative care’. 

NCVER Deputy Director Peter Thomson spoke on the role and impact of flexible delivery, while 
Jessie Borthwick and I gave papers on the nature of VET in Australia at that time, examining the 
statistics. This conference agenda brought together a lot of government leaders, including the 
Honourable David Kemp, the federal minister, researchers and policy people, probably for the first 
time in the VET sector. 

The second part of the conference program focused on industry perspectives, with Steve Balzary 
and Kevin Peoples giving quite different perspectives on industry’s training needs. Stephen Billett 
spoke about some of the research he’d been doing on returns to enterprises from investment in 
training and VET. Chris and Joy Selby Smith spoke on the user choice policy that was emerging. 

The third part of the program, looking at VET students, was opened by the Honourable Dorothy 
Kotz, the then South Australian state minister for training. At that time there was tremendous 
debate in Australia about whether VET was for students or industry. Sandra Pattison gave a paper 
on the student profile. 

I think it’s useful to look back at that time — this was when the VET research world was beginning 
to venture out into other areas and undertake work with some direct relevance to the major policy 
debates at the time. 

The next national VET strategy took a step further and an attempt was made to line up the VET 
research priorities with the objectives of the ANTA national strategy, developed after extensive 
consultation with providers, stakeholders, industry parties and states and territories. This was a 
major effort, complicated by the fact that everyone had a completely different view of what the 
objectives of VET ought to be. NCVER subsequendy developed a research strategy that attempted 
to accommodate those national policy objectives and this determined the direction of the research 
work for a number of years. Much the same sort of process occurs today, with NCVER staff 
talking with states and territories and other stakeholders and reaching an agreement about the 
research agenda for the coming period. 

At that time NCVER became aware of the importance of disseminating research — getting the 
message out to the wider community. NCVER has developed some innovative strategies in 
attempting to get the research message out, including to stakeholders, and NCVER is one of the 
more sophisticated research agencies around the world in this regard, expending a great deal of 
energy informing stakeholders on the results of relevant research. Public research forums, which 
focus on industry people and other members of the wider VET community, are a case in point. 

There are a number of specific areas of NCVER research which I consider have had a direct impact 
on policy and policy thinking; for example, work done in the last few years on returns to 
investment in training, while research on apprenticeships and traineeships was widely disseminated 
through public forums and given extensive media coverage, consequently achieving direct results in 
terms of policy developments in the apprenticeship system. 

Group training is another example of where NCVER research has informed VET policy and 
direction. NCVER analysis of the impact of group training was fed into a national review, 
demonstrating to the entire VET sector that group training was having a positive impact on 
apprenticeship training around Australia. The usefulness of the student and graduate surveys has 
led to the development by NCVER of ‘a do it yourself kit’ to enable TAFE teachers and other 
VET trainers to survey their own clients in terms of the effectiveness of what they’re doing. 
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Skill shortage work begun by the then federal minister David Kemp, recognised that NCVER had 
already done a great deal by 1998 in projecting the oudook for training in Australia’s industries. In 
1999 when the minister commenced an industry-led process of looking at skills shortages in a 
whole range of industry areas, NCVER was asked to come in and provide a factual and analytical 
basis to that work. This turned out to be very important at the time, since it identified that skill gaps 
did exist in some industries. 

Work recently undertaken in Queensland on a new skills package for the VET sector drew 
extensively on the work of NCVER, from both the research and statistical areas, and of course it 
also drew on the extensive statistics produced by the Australian Bureau of Statistics (ABS). The 
point is that there was a strong research basis in this work, but it also involved community 
consultations. 

In relation to the role and value of the impact of research on policy, it’s not usually a one-to-one 
linear relationship between one research project and one consequent policy change. The process 
more resembles ‘osmosis’, where gradually the research findings and the analysis filter down to 
become part of the knowledge base of people working in the sector and people planning in the 
sector. In this context I would argue that undertaking a range of research projects in an area, 
rather than a single project, has a far more significant impact, especially when that work is 
coming from different angles and offering differing perspectives, in terms of conclusions. The 
more varied the research and its viewpoints, the more effective it is in terms of adding to the 
knowledge base and promoting debate. 
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Discussant: Peter Kearns, 

Global Learning Services 



In the context of the very impressive overview that Chris Robinson has given us of the 
development and the increased sophistication of VET research since the early days of NCVER, I’d 
like to pick up two issues that weren’t very much evident in what Chris had to say but which I think 
we do need to consider. The question is: How do we achieve an optimum interaction between 
research policy and practice while developing systems that are truly and richly interactive and 
adaptive to changing circumstances? 

In developing this point, I’ll comment on two contexts, the community and regional context and 
secondly, the international context. Bearing in mind that the national strategy that Chris referred to 
is directed at VET serving employers, individuals and communities, it is however in the context of 
communities where we encounter the issue of the enormous diversity that exists in communities 
across Australia. How therefore can VET be effective in that context? Much VET policy has been 
developed according to national and state priorities. Rarely have we got down to the level of 
communities and considered the myriad of influences that affect outcomes in communities. By 
influences in the community I mean the culture, the history, the heritage, the industry base and so 
on. And if we are concerned with the role of research and with outcomes, then I think we have to 
‘grapple’ with what happens in communities. 

In approaching this, I’d like to leave with you some findings from the Organisation for Economic 
Co-operation and Development (OECD) in relation to communities. This is work that was done 
on regional competitiveness and skills. The OECD saw three factors that were important in that 
context: firsdy, the region as an area for dialogue and cooperation; secondly, the region as an area 
for policy integration; and thirdly, the region as an area for negotiation. 

When you look at regions across Australia, as we have been doing recently in some research, you 
see enormous diversity in the extent to which these conditions operate and you see great barriers to 
these conditions; for example, competition policy and the nature of funding and how these impact 
on collaborative practices in regions and in communities. 

I would add the region to the OECD list as an area for innovation and for building a learning 
culture, both of which I consider to be important, particularly in the context of the realities of 
global competition. Policy integration of course is an important area, and the achievement of 
‘joined up’ policies — the extent to which what happens in education connects with what happens 
in health and in welfare and so on — is crucial. And it’s largely at the community and regional 
levels where these different spheres have to come together. In this context therefore should VET 
research be far more concerned with what happens at these levels? 

The second context, the international context, is important for a number of reasons: firstly, as a 
source of new ideas as a stimulus to innovation in a world of constant change. I wonder how well 
known in Australia are interesting and important developments in other countries and how much 
impact they have on policy; for example, the initiatives in Canada at the present time with the new 
Canadian Council on Learning and its work on a composite learning index. This has been applied 
to communities right across Canada, and the range of development in those communities in terms 
of learning capability is evident as a base for benchmarking and for improving performance. Also 
from Canada is the work being done on the status of learning. So these developments and 
initiatives in other countries — our economic competitors — are in a sense setting benchmarks for 
Australia that we need to be aware of. The second reason why we need to be concerned, perhaps 
more concerned with the international context, is that it’s a framework for understanding the 
competitive pressures that Australian industry is subject to. 

A recent and hard-edged report from the New Commission on the Skills of the American 
Workforce, called Tough choices or tough times , is very much concerned with the changing global 
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environment currently confronting American industry. The report highlights the rise in the 
numbers of well-educated, highly skilled people in countries like China and India, and the 
consequent temptation faced by American industry to export jobs off shore. The commission 
estimated that 16% of jobs were at risk and vulnerable. This is obviously an enormous proportion 
of the American workforce and, if combined with all the jobs at risk because of automation, then a 
very serious challenge is presented. 

No Australian research or policy documents at this time are producing similar, very frank 
assessments of the realities of the global and technological context in which Australian industry 
operates; hence, the challenge for preserving jobs. The implications are that we must raise the skill 
level of the Australian workforce and build a high- skill equilibrium with the capacity in the 
workforce to be creative and innovative. However, there are enormous barriers to be overcome in 
tackling this challenge — the low levels of literacy in the adult population in much of our workforce, 
the cultural attitudes in communities and much of industry towards learning, and the limited extent 
of development of lifelong learning policies in Australia. 

I’d like to return to what OECD had to say and to suggest ‘the region’ as areas for dialogue, 
cooperation, policy integration, negotiation and as arenas for innovation and for building a 
learning culture. 

In terms of the role of research, I see it as a lighthouse illuminating the sky, ocean and land, or if 
you like, past, present and future, with a flashing beam. This is an important role. I’d then like to 
suggest that we think in terms of a quadrant in which the four parts would be: looking in; looking 
out; looking back; looking forward. I think it’s true to say that much VET research has been in the 
‘looking in’ quadrant. It’s been research that’s concerned about the nuts and bolts and features and 
outcomes of the system. However, if the sector were to adopt an outward-looking perspective, then 
the sorts of research areas chosen might include the implications of demographic change, the 
ageing of the population, the implications of globalisation, and so on. 

Chris gave us an excellent overview of past events (my looking-back quadrant) but now, looking 
forward, the question is — have we got the balance right? Is the balance in research appropriate in 
those areas? Do we need to re-think the current VET paradigm and work towards one relevant to 
the twenty-first century context and the challenges confronting industry in Australian society? My 
answer is yes. This is also the conclusion of the UNESCO second international congress on VET, 
the joint UNESCO— ILO 2 recommendation on Technical and Vocational Education for the 
twenty-first century, the UNEVOC 3 International Experts meeting in 2004 on TVET and 
Sustainable Development and the recent UNEVOC discussion paper, ‘Orienting technical and 
vocational education and training for sustainable development’, as well as the work of OECD over 
more than a decade. 

In re-thinking the VET paradigm for the twenty-first century context of VET, we should have 
regard to a recent conclusion from the report, A well-skilled future-, that the demand for VET skills 
is likely to evolve in two ways — towards higher-level qualifications and towards more interactive 
and cognitive skills rather than motor skills. The pedagogical implications of this demand serious 
consideration. 

So if I may conclude with a sentence from the recent UNEVOC discussion paper, ‘Orienting 
technical and vocational education and training for sustainable development’: 

Unfortunately TVET in many countries remains locked in the role of being a supplier of 
skilled labour to industry and it’s therefore unable to respond effectively to the needs of the 
emerging information age. 



2 ILO — International Labour Organization. 

3 UNEVOC = International Centre for Technical and Vocational Education and Training. 
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And I would add to that the needs of communities for sustainable development, so what can 
research do? I’ll leave that question with you. 
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Developing and sustaining a 
VET research capacity 



Berwyn Clayton, 

Canberra Institute of Technology 



Introduction 

Quality research has considerable potential to improve policy and practice in vocational education 
and training (VET). The 1992 report No small change demonstrated this potential by providing the 
evidence base needed to generate greater interest and commitment to research in the sector. 
Additional resources encouraged the development of the critical mass of university-based 
researchers required to sustain a strong program of research focused on national VET priorities. 
Furthermore, the national VET research competitive grants programs managed by the National 
Centre for Vocational Education Research have supported a broader community of researchers, 
built the stock of VET knowledge and greatly enhanced the status of Australian VET research. 
However, this research capacity is now under some threat. There are thinning numbers in university 
centres and the population of senior researchers is ‘greying’. Research grants for university 
researchers are more generous outside the sector, while for researchers within the training system, 
research has been largely redirected to support quality and compliance. To sustain and build VET 
research capacity, the entire VET research community will need to find ways to attract and nurture 
new and emerging researchers into the field. Individual researchers and research organisations will 
also need to work much harder at raising the profile and value of VET research. 



Looking back: The emergence of a VET research community 

No small change-. Providing die foundation 

In 1992, the Vocational Education, Employment and Training Advisory Committee commissioned 
report No small change-. Proposals for a research and development strategy for vocational education and training in 
Australia (McDonald et al. 1992) found that research on the VET sector was fragmented, poorly 
focused, limited in its critique of policies and programs and not fully utilised by those who could 
best benefit from the outcomes of the research focus. 

In this seminal report, McDonald and his colleagues highlighted the major weakness in the existing 
research environment and offered a raft of recommendations designed to strengthen the role of 
research and researchers in supporting the training reform. First and foremost, they identified the 
need for a national research strategy, underpinned by a set of principles relating to priorities and 
funding, as well as the organisation and utilisation of research and development. No small change 
argued for a more strategic approach to research and emphasised the importance of focusing on 
areas of high priority to the sector, on improving the quality and quantity of research and enhancing 
the means of its dissemination. For the authors, the reasoning behind their recommendations was 
quite clear: 

A stronger research effort will benefit the sector by providing a better information base, 
critical analysis and accountability, improved cost effectiveness, varied perspectives, a better 
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understanding of education and training processes and a higher profile for vocational 
education and training. (McDonald et al., p.v) 

At the same time, McDonald et al. identified a number of requirements that were vital if VET 
research was to achieve any of the agreed goals. In the first instance, research needed to draw on 
the varying perspectives of research, institutions and other key stakeholders directly involved in 
vocational education and training. Equally important were the forging of closer links between 
researchers, policy-makers and practitioners and the encouragement of university researchers from 
different disciplines and fields to carry out research in the sector. Greater collaboration, particularly 
at the local level, between industry, TAFE and university researchers was also desirable, as was the 
allocation of resources to achieve a ‘critical mass’ of researchers and research projects in areas of 
high priority. 

Training was seen as a key strategy in cultivating this ‘critical mass’ and McDonald et al. proposed 
a number of approaches. These included, among other things, specially developed postgraduate 
courses for VET practitioners, opportunities for apprenticeships with experienced university 
researchers, scholarships to undertake relevant postgraduate programs of study and university- 
practitioner collaboration in action research to improve the quality of the research and increase its 
uptake in day-to-day vocational education and training practice. 

Thus, greater collaboration and training were seen to be critical for the development of a 
community of researchers capable of providing the evidence base required to support the national 
training reform agenda. But, in proposing strategies and initiatives to increase the effectiveness of 
research dissemination, McDonald et al. made an important recommendation that was later to have 
a significant impact on developing the ‘critical mass’ that they were seeking. They noted: 

One step that would greatly improve networking ... is the establishment of a single 
association of vocational educators, trainers and researchers. This could be achieved by the 
broadening of an existing network ... or starting a new association (p.72). 

The ANTA Research Advisory Council 

In response to the recommendations of No small change, the ANTA Research Advisory Council 
(ANTARAC) was established in 1993. During the period 1994-96, the council managed the 
national program of VET research. As well as identifying research priorities, funding was provided 
for the establishment of key research centres. This money was directed to the Centre for the 
Economics of Education and Training (Monash University and the Australian Council for 
Educational Research) and the Research Centre for Vocational Education and Training (University 
of Technology, Sydney) and for research partners — the Centre for Post-compulsory Education and 
Training Research (University of Melbourne and the Royal Melbourne Institute of Technology) and 
the Centre for Research into Learning and Work (University of Tasmania). Each of these centres 
put together comprehensive programs of research and afforded the essential focus for an 
aggregation of researchers in these universities to work together on many of the key priority areas 
for the government’s training reform agenda. 

The ANTA Research Advisory Council also funded research projects, scholarships and innovative 
training activities. These activities, together with centre and research partner funding, proved to be 
critical to the building of research capability in the sector and to the development of the VET 
research community in Australia. There was, however, only limited networking between these 
centres and others engaged in VET research in the sector, particularly those focusing on research 
into VET policy and practice in state-based vocational education and training agencies, TAFE 
colleges or in industry. It was with the arrival of the Australian Vocational Education and Training 
Research Association (AVETRA) that the extent and potential of this broader research community 
came to be better understood. 
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A single association of vocational educators and researchers 

The Australian Vocational Education and Training Research Association was established in 1997 
with an initial grant from ANTA Research Advisory Council of $10 000 and additional support 
from the National Centre for Vocational Education Research. Flagged as Australia’s only national, 
independent association of researchers in vocational education and training, its membership is open 
to any individual who has an interest in VET research. 

The goals of the Australian Vocational Education and Training Research Association are to: 

A further the contribution of VET research to the development of Australian VET policy 

A independently review VET research priorities 

A promote independent and significant research in VET 

•y" raise the profile of VET research in the academic and training communities 

A promote training in research methods for those working in or with the VET sector 

A provide platforms for the dissemination of research findings (AVETRA website, 

<http:/ /www.avetra.org.au/ about/index.shtml>). 

In the ten years of its existence, the organisation has maintained a steady membership of around 
200, although the number of active members is somewhat less than this figure. But it is the make- 
up of the association’s membership that best reflects the changing nature of research in vocational 
education and training. Members include not only university-based researchers but also 
practitioner— researchers and practitioners from public, private and enterprise registered training 
organisations; training managers from major corporations; VET policy-makers and researchers 
from state and territory-based agencies; industry skills council representatives; and consultants. 

Such an array suggests that the association has been able to achieve its goal of raising the profile of 
VET research across the sector, a situation that has been made possible through successful annual 
conferences and state-based October VET events across the country. 

While capacity building has been a long-standing imperative for the association, such activities 
require commitment from the membership and the broader VET community — policy-makers, 
funders and researchers alike. 



NCVER: Raising the profile, making the connections 

Since 1997, NCVER has played a significant role in building Australia’s VET research capacity by 
contributing to ‘the growth of an accessible stock of knowledge and research on vocational 
education and training’ and ‘the human capacity to undertake and use VET research’ (KPA 
Consulting & Phillips Curran 2003, p.5). 



Managing knowledge, sharing knowledge 

NCVER’s vision is ‘for policy and practice in vocational education and training to be based on 
sound evidence’ (NCVER 2007, p.2). In line with this vision, the organisation manages the national 
competitive grants program, undertakes strategic in-house research and collects and analyses 
statistics and survey data. The outcomes of these activities are made freely available to government 
agencies, training providers, peak industry bodies, enterprises and individuals. National and 
international sources and resources are also available via the world-class research database, VOCED. 

Such a vast collection of knowledge, most of which is readily accessible and down-loadable is an 
invaluable resource for VET researchers, students of vocational education and training, as well as 
practitioners, planners and managers in the field. This is particularly true when clusters of projects 
are generated around themes which are of critical interest to specific stakeholder groups. Good 
examples of these are the collections of research around return on investment in training, generic 
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skills, workplace learning, learning styles, flexible learning and literacy and numeracy activity in the 
VET sector. The power of this themed approach is that the accumulation of knowledge around a 
topic offers diverse and sometimes multi-layered perspectives which are attractive to those who 
want answers to the problems they might be facing in their day-to-day work. 

The bodies of work produced by the two NCVER-managed consortium research programs have 
similar potential. The outcomes of ‘A well-skilled future: tailoring VET to the emerging labour 
market’ should provide policy-makers, peak industry bodies, training providers and employers with 
a sound foundation for policy development, planning and targeted training. For training providers 
and policy-makers, evidence from ‘Supporting VET providers in building capability for the future’ 
should support decision-making about many of the critical workforce issues currently facing 
registered training organisations across the sector. 

Utilisation of research, however, is very much about changing mindsets, and ease of access to 
research findings is a crucial factor in uptake. 



Dissemination, brokerage and support 

In writing about how educational research can best make an impact on educators, Figgis et al. 

(2000, p.366) suggest that: 

. . . research has to be propelled out of its abstract conceptual space and into an arena where 
educators [and policy makers] can engage with it and decide if and how it will be useful to them. 

For the last decade, NCVER has used multiple strategies to bring research into the many arenas 
where potential end-users reside. Acknowledging the diverse needs of the sector, publications like 
Research messages , Insight and At a glance are designed to provide key information in summary form, 
while the NCVER’s website (<www.ncver.edu.au>) allows electronic access to full research reports 
and the range of summarised information. 

In addition, NCVER conducts state-based forums on key priority areas and brokers research 
findings to targeted stakeholder forums and briefing sessions to ensure connections are made with 
those who can best use the outcomes. Identified by Robinson (1999, p.14) as ‘a core and critical 
priority for NCVER’, this brokerage role continues to be a paramount activity in building research 
capacity in the sector. 

Of equal importance to emerging researchers is the annual NCVER National VET Research 
Conference. For many years this conference has been the stepping-off point for novice VET 
researchers, particularly postgraduate students and practitioner— researchers. Now in its sixteenth 
year, this ‘no frills’ forum offers the opportunity to make connections, test ideas, present practice- 
based action research, trial research methods and tap into the wisdom of more experienced 
researchers in an open and low-key environment. 

The extent to which NCVER has engaged in capacity-building for the VET research community is 
best summed up in the following statement from the Australian Vocational Education and Training 
Research Association submission to the 2006 review of NCVER’s research and statistical services: 

NCVER has supported and promoted the activities of AVETRA in a positive and meaningful 
way through publications, statements and the willingness of NCVER staff to engage with and 
support VET research and VET researchers 

(Australian Vocational Education and Training Research Association 2006, p.3). 



Building and sustaining research capacity: The challenges 

Clearly great headway has been made in VET research since the days of No small change , with more 
dollars being dedicated to the activity and more research being undertaken across a range of priority 
areas for the sector. But is this effort sustainable? A close look at the state of the community of 
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VET researchers and the environment in which it operates would suggest continuity and 
sustainability are an issue. 



The community of VET researchers 

Given the diverse nature of vocational education and training, it is difficult to ascertain with any 
degree of accuracy the numbers of researchers working in the sector. It would be appropriate to 
assume that the largest proportion is located in universities, with a smaller number in TAFE 
institutes and NCVER itself. An unknown number are working as consultants, or in state and 
territory training authorities, government departments and industry. 

In the period 1997 to 2002, 218 projects were funded through the national VET research 
competitive grants program; 53% were conducted by university-based researchers, 27% by 
consultants, 11% by TAFE-based researchers and 10% by researchers in government departments 
and others. Gibb (2003 unpublished) estimated that among the 218 projects, approximately 10% 
were conducted by principal researchers 40 years of age or younger, while approximately 43% were 
carried out by principal researchers over 50 years of age. Gibb noted that reputation and experience 
required time to establish, and therefore it was unsurprising to find such a high percentage of 
principal researchers in the older age groups. 

From 2001 to 2005, a total of 226 researchers conducted 186 projects under the national VET 
research competitive grants program. Fewer projects were undertaken by researchers in TAFE and 
government departments; however, 23% were done by consultants and 69% by university 
researchers, highlighting the dependence that the sector has on higher education for research input. 
As in previous years, the majority of principal researchers were probably above 50 years of age. 

The ‘greying’ of the VET research community and the subsequent erosion of critical mass, however, 
is not the only issue impacting negatively upon the maintenance of viable numbers in VET research. 
Attracting new or emerging researchers to the field and retaining the ones already there have become 
key issues for funding bodies, research centres and the Australian Vocational Education and Training 
Research Association alike. Opportunities for mentoring appear to be quite limited outside 
postgraduate programs, as are training programs for practitioners working in the field. 

Without the introduction of new people into the research community to replace those who are 
leaving, together with targeted training and the ongoing support of highly experienced mentors, the 
VET research community would seem to be in a highly vulnerable position. Sustaining Australia’s 
VET research capacity under these circumstances will require considerable commitment of time, 
energy and resources from key players in the sector. 

Research environment: Infrastructure and culture 

NCVER lists 20 research centres on its links page — the majority of which reside in universities. The 
amount of VET research these centres undertake, as would be expected, is dictated by the availability 
of funding. Currently there is insufficient funding available through most National Vocational 
Education and Training Research and Evaluation projects to sustain even the smallest of these 
centres. As a consequence, centre directors must identify alternative sources of research funding to 
meet internal university financial targets and ensure the ongoing survival of their centres. Grants 
from sources beyond those in vocational education and training not only offer opportunities to build 
and sustain research capacity, they promote the redirection of research interest and effort elsewhere. 
As an outcome, the critical mass of researchers focusing on VET no longer really exists within the 
majority of these university centres, and any opportunities for the development of Research Quality 
Framework-promoted research concentrations specifically directed at exploring policy or practice in 
the sector are likely to be few, if any. 

Within the sector itself, a similar process of diminution and redirection of effort has been in 
evidence in TAFE systems for a considerable time. Research within training providers and state and 
territory training authorities is largely directed at fulfilling business and political imperatives and is 
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conducted to support quality systems, compliance requirements, planning and market research. In 
parallel with these changes, a lack of resources has diminished the amount of teaching and learning- 
focused practitioner action research occurring across the sector. 

These difficulties associated with developing capacity are exacerbated by a culture in vocational 
education and training in which research is not generally valued. Practitioners are only rarely 
encouraged to dedicate time to reflecting on their practice or the research that might impact upon it 
in a systematic way. While Reframing the Future, with its Goal 5, Applying information and research, 
goes some way to addressing this issue, its reach and impact are not broad-ranging. 

Despite the best efforts of NCVER, VET research organisations and individual researchers, a 
similar situation exists with key decision-makers in training providers and those in organisations 
overseeing the vocational education and training system (Taylor 1999; KPA Consulting & Phillips 
Curran 2003; Department of Education, Science and Training 2006). While debates continue over 
quality, utility and timeliness, this lack of identity or visibility for research means that it may not be 
considered in the formulation of policy. Any disconnection between research and policy-makers 
limits the potential that research might have to broaden existing perspectives and explore 
alternative approaches. And where effective connections have been made, the constant whirlpool 
of change that exists in the sector means ‘the connections and networks needed for the transfer of 
research to policy have been increasingly difficult to sustain’ (Kell 2005, p.5). 



Looking forward: The future of VET research in Australia 

To ensure that research capacity is further developed and sustained, different strategies will need to 
be developed to address the challenges facing VET research. This will require the combined 
energies of NCVER, the Australian Vocational Education and Training Research Association and 
the research community as a whole. Needless to say, such strategies will need sympathetic policy 
and funding support. 

NCVER: More of the same and then some 

As the primary VET research organisation in Australia, NCVER’s role in developing and sustaining 
VET research capacity is a significant one. Importantly, many of the strategies for building capacity 
are already in place. These include broad-ranging consultations, close communication with key 
stakeholders, quality dissemination of research using a range of media and the conduct of activities 
for sharing and applying research outcomes, all of which raise the profile of VET and confirm the 
value of research to the sector. 

NCVER also has the capacity, as a research centre conducting research in its own right, to attract 
new researchers to the organisation, quickly building their understanding and experience of the 
sector through mentoring and training. But it is the pool of experienced researchers within NCVER 
that is perhaps the greatest asset to the research community at a time when more and more 
researchers are moving out of VET research. Such a resource could be used for building capacity, 
particularly in mentoring novice researchers, providing technical advice, conducting workshops on 
research methods and supporting research networks in key priority areas. Another option would be 
to offer opportunities for inexperienced researchers to undertake a brief ‘cadetship’ or shadow a 
senior researcher in the organisation. The advantage of these strategies for NCVER would be to 
retain closer connection with practitioners and researchers in the sector. 

In relation to the management of the National Vocational Education and Training Research and 
Evaluation program, NCVER has already established the value of collaborative research activities 
which utilise the expertise and differing perspectives of groups of researchers drawn from diverse 
organisations. With reduced numbers of VET researchers in universities, it would seem most 
appropriate to continue the concept of networked research activities, particularly around priority 
areas. Linkages between researchers from different locations, organisations, perspectives and 
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disciplines may continue to form through naturally occurring relationships. However, it may also be 
appropriate for NCVER to encourage researchers with specific priority area expertise to register 
their interest as a network for particular projects or group projects. Such an approach may mean a 
proportion of each round of funding being dedicated to commissioned work undertaken by 
networks with preferred provider status in particular areas. The challenge will always be to provide 
research networks with the stability and flexibility they need to sustain their research effort. This 
requires longer-term funding and continued support and encouragement for research networks. 

It is also acknowledged that participation in National Vocational Education and Training Research 
and Evaluation projects must be broadened to include many more researchers than the 
‘experienced and respected’ mentioned in the recent review of NCVER’ s research and statistical 
services (Department of Education, Science and Training 2006, p.12). While scholarships, 
traineeships and fellowships might be difficult to resource, strategies that encourage researchers to 
involve new researchers in projects are a step in the right direction. A potentially more powerful 
approach would be to insist that this occurs in the majority of National Vocational Education and 
Training Research and Evaluation projects. 



The Australian Vocational Education and Training Research Association: 
Building commitment to build community 

The Australian Vocational Education and Training Research Association, as an independent 
association, has an opportunity to play a key role in developing Australia’s VET research capacity 
by addressing some of the issues of sustainability. In fact, the association’s very survival may well 
be dependent upon its doing so. With annual conferences, state and territory-based October VET 
activities, the journal and newsletters, the association is well placed to raise the profile of VET 
research. It is also in the position to actively recruit new and potential researchers, facilitate 
connections between leading and emerging researchers, encourage knowledge-sharing, sponsor 
programs to build the research capacity of university students and VET practitioners, and conduct 
activities for sharing and applying research outcomes. 

In addition, members of the association are well placed to collect information about postgraduate 
courses and research skills training across Australia’s universities and to build the skulls and 
confidence of novice researchers through internal individual mentoring, training and technical 
support. 

The research community as a whole: Communicating the value of research 

Research capability building can be fostered or discouraged by shifts in prevailing policy. Policy can 
either facilitate or deter collaboration between researchers. It can provide the environment in which 
developmental opportunities for researchers can flourish, or it can threaten the sustainability of 
supportive structures like research centres, research networks and even professional associations. It 
is imperative, therefore, that the research community as a whole focuses on communicating the 
importance of VET research to the sector by: 

A building linkages with key policy-makers at local, state and federal levels 

A developing the political skills to navigate a place for research evidence in policy formulation, 
program implementation and review 

A developing strategic partnerships with VET organisations like the TAFE Development Centre, 
the Australian Council for Private Education and Training, and TAFE Directors Australia 

A strengthening grassroots connections with potential end-users of research, particularly in the 
realm of practice. 

By restating the importance of quality research to policy and practice in the sector, it is more likely 
that a case can be made for sustained and growing investment in the activity. 
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Conclusion 



In the aftermath of No small change, there was a considerable growth in interest and action in VET 
research in Australia. With an injection of additional funding, a national strategy, a thriving 
NCVER, a strong and independent association of researchers and 20 research centres, Australia’s 
VET research capacity was well developed. In recent years, the critical mass of university 
researchers drawn together by ANTA’s key centres has diminished considerably because of natural 
attrition and the attractiveness of other funding sources and different research directions. For 
NCVER, the Australian Vocational Education and Training Research Association, research centres 
and individual researchers, a major challenge is now to attract and nurture new researchers. 

Another is to continue to build the vital connections with those who can best use the results of the 
research endeavour — policy-makers and practitioners. As with No small change, the time is now right 
to re-emphasise the considerable potential that research has for quality policy and practice in the 
VET sector. 
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Discussant: Andrew Smith, 
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Berwyn Clayton’s paper on developing and sustaining a VET research capacity raises a number of 
interesting issues for the future of VET research in Australia. As Berwyn makes clear, thanks largely 
to the funding made available through the Australian National Training Authority and latterly, the 
Department of Education, Science and Training, since the early 1990s, VET research has flourished 
in Australia to the point where it ranks as a world-class research activity. Few, if any, other 
countries have developed such a focused and consistently funded research program in VET. But it 
is in danger of diminishing rapidly for the reasons that Berwyn has outlined in her paper. Essentially 
VET research has hit a classic skills shortage situation — a rapidly ageing workforce and not enough 
younger researchers coming through to take the place of retiring researchers. However, as Berwyn 
acknowledges in her paper, in the case of university-based VET researchers, who comprise the 
majority of researchers in this area, the situation is exacerbated by recent developments in national 
research policy and its impact on university research structures. The question is what to do about 
this. I want to pick up on three themes which arise from Berwyn’s paper and which may provide 
some possible responses to the VET research skills shortage. 

The first is networking. Berwyn is quite right to highlight the importance of networking amongst 
VET researchers as a means of boosting capacity. When the research community is shrinking, it 
makes sense for the remaining researchers to work more closely and purposively together. Berwyn 
referred to the research centres listed on the NCVER website that carry out VET research. Ten years 
ago the NCVER counted some 20 centres dedicated to VET research throughout Australia. These 
had largely been brought into being by the increased funding from the mid-1990s. Now most of 
these dedicated centres have disappeared with only one or two honourable exceptions. Today’s 
centres undertake VET research usually as a sideline to their mainstream research activities and foci. 
Research centres are a good way to develop research capacity as they encourage younger researchers 
and sponsor research students. But the traditional notion of the research centre may be problematic 
in that it can be exclusive to a particular place and rarely include all the best researchers in a field. In a 
sparsely populated research community such as the VET research community, too many centres can 
simply spread the talent too thinly, reducing rather than enhancing capacity. One way to counter this 
effect is to think of centres as networks of researchers rather than as places where they work. The 
example of the Economic and Social Research Council’s Centre on Skills, Knowledge and 
Organisational Performance (SKOPE) in the United Kingdom is useful here. SKOPE has had a far- 
reaching influence on thinking and policy-making in VET not only in the United Kingdom but 
around the world. It has achieved this by linking all the best researchers in the United Kingdom, 
Europe and in the United States in a networked centre. NCVER has had some success recently with 
research consortia, which could be seen as precursors of fully networked research centres. I think 
there is scope (pardon the pun) for us to think more carefully about more permanently funded 
networked VET research centres in Australia along the lines of the SKOPE centre. 

The second point is leverage. Berwyn rightly observed that university-based researchers are under 
pressure to increase their research funding from all sources. This can be a major distraction from 
VET research as researchers seek Australian Research Council and other funding sources to 
maintain their position within universities. But the search for diversity in research funding can also 
provide opportunities for VET research. Australian Research Council linkage grants provide a good 
example of leveraging available research funding to advance VET research. Many VET institutions 
can be partners on linkage grant applications — state training authorities, skills councils, employer 
bodies, unions — even the NCVER itself has had some involvement. My sense is that we could 
make more of this type of leveraged funding if more VET researchers were encouraged to take the 
linkage route. Perhaps this might be a role for NCVER as keeper of the national research priorities 
in VET — to work with VET researchers to produce research proposals with suitable partners and 
which have a high probability of gaining Australian Research Council funding. But leveraging is not 
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confined to Australian Research Council schemes. The bodies that want more research could, over 
time, perhaps work more closely together to fund research in VET that would push the envelope of 
research funding past the $2—3 m of annual National Vocational Education and Training Research 
and Evaluation program grants into something much larger and more strategic. 

Finally, internationalisation. Just as it has been a traditional response for Australia to look overseas 
during times of skills shortages, perhaps we could look internationally to increase our domestic 
VET research capacity. VET has clearly become increasingly internationalised in recent years. But 
this is not just about the export of education. The problems that we face with skills and training in 
Australia are very similar to those experienced in other developed countries. We often look 
overseas to learn from the experiences of other countries, and other countries look to Australia. 
There is a powerful two-way exchange of ideas for improving VET policy and practice. But our 
research tends to be very national in scope. By comparison, VET research in Europe is highly 
internationalised, as one might expect through the European Union, and this has led to the creation 
of international networks of VET researchers working on larger, high-quality research programs. 
Australia could benefit by becoming more integrated into these international networks. 

International funding is now becoming more available for Australian VET researchers. The 
Australian Research Council has joint programs with both the United Kingdom Economic and 
Social Research Council and with the European Union. Australian researchers can participate in 
well-funded research programs such as the European Union framework program, as third country 
partners, and are also accessing American National Academy of Science grants in collaboration with 
United States researchers. International cooperation of this kind can both enrich VET research in 
Australia, bringing new perspectives to complex problems, and increase the pool of VET research 
expertise upon which we can draw. 

I want to thank Berwyn for an interesting and stimulating paper and I hope my comments might 
help us move towards some possible solutions to the VET research skills shortage. 
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Introduction 

Australia has a long history of gathering statistics on education. In common with most other 
statistics of the pre- and early Federation period, the primary source was administrative records 
maintained by individual states, which were then aggregated to report national statistics. What is 
distinctive about education and training statistics — viewed across schools and higher education, as 
well as vocational education and training (VET) — is how heavily reliant they still are on 
administrative collections. In many other domains of social and economic life, administrative 
collections have yielded to household and business surveys and the five-yearly census, most of 
these conducted by the Australian Bureau of Statistics (ABS). 

The ABS runs two major household surveys on education and training. The Survey of Education 
and Training, first conducted in 1989, then each four years since, and the annual Survey of 
Education and Work are extremely valuable and rich data sources for understanding patterns of 
participation (and non-participation) and educational attainment, and the relation of both to 
measures such as employment and earnings. The census and various other specialised surveys — 
those relating to disability and adult literacy, for example — also contain much valuable information. 

However, the ABS is unable through its surveys to meet the information needs of the VET sector — 
more so than any other sector of education. One example will illustrate the point. Educational 
attainment, as measured by the ABS, is based on qualifications that have been completed; yet the 
norm in the VET sector is for students to complete only part-qualifications. ABS measures of 
attainment in any given period, therefore, will significantly understate the true quantum of skill 
development that has taken place. Even if sample surveys were capable of getting to grips with the 
nuance of the sector, it remains the case that the information demands extend well beyond the 
acceptable error limits of even large sample sizes. That is, there is considerable demand for small 
area and small group statistics. A typical example would be the number of Indigenous students 
enrolled at certificate II level for each state and territory. Only administrative collections are able to 
meet such narrowly defined data requests. 

The difficulty of constructing or designing an administrative collection that can adequately cater for 
both nuance and fine detail should not be underestimated. Indeed, the development of national 
training statistics is a tale that can broadly be characterised as one of satisfactorily resolving the 
tension between, on the one hand, a national desire for comparable (across time and space) 
statistics on the activities of technical and further education (TAFE) institutes and, on the other, 
the fragmented, inconsistent and incomplete records maintained by TAFE authorities. It was a long 
time in resolution and, as we will go on to argue, the work is not fully done. Grant and Fisher 
(1994) provide a detailed account of the slow development up to 1994, impetus coming from 
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national developments book-ended by the report of the Kangan Committee in 1973 and the 
national training reform agenda of the early 1990s, but interspersed by long periods of lull, with 
national committees unable to reach agreement on matters such as purpose and cost. 

It is not our intention to pick up where Grant and Fisher left off and recount the history of those 
developments since the early 1 990s, although some history is needed to understand why things 
have developed as they have. Instead, we offer two observations, observations which will provide 
the basis for the remainder of this paper. First, the considerable progress made over the last decade 
or so can largely be attributed to the design of statistical standards which accommodate a range of 
interests — national, jurisdictional and providers themselves — and through governance 
arrangements which allow for these to evolve and adapt over time to changing and emerging 
information requirements. Second, while the VET sector might be said to be ‘moderately well- 
served statistically’ (ABS & NCVER 2002), and to be of good standing internationally, there are 
a number of embedded flaws in the underlying design which limit the utility of the statistics in 
meeting public accountability and monitoring requirements and for undertaking applied policy 
analysis and evaluation. Some future directions are suggested in our conclusion. 



Why do we have national training statistics? 

To meet the needs of TAFE administrators and state training authorities a range of state-based data 
collection systems had evolved over time. These systems were non-comparable in many respects, 
which hampered or prevented national training statistics being produced. 

The rationale for stitching these various systems together arose as part of the push to create a 
national VET system out of the existing federated system. At much the same time as the Australian 
National Training Authority (ANTA) was established in 1992, the Australian Government made its 
funding conditional on data collection, reporting and accountability arrangements. These 
arrangements, agreed to by states, were much stronger than had existed previously and led direcdy 
to the national collections of VET provider and financial information, managed by NCVER under 
the direction of the National Training Statistics Committee and its predecessors. It was this 
development, more than any other, which broke the impasse noted by Grant and Fisher. 

It is true that new funding agreements did not in themselves create an imperative for national data 
collection arrangements, as reporting requirements could have been met through specific 
arrangements with each state. However, the legislation which established ANTA included a 
statutory obligation to develop key performance measures and report these annually to Parliament. 
The requirement to report measures, which were consistent and comparable across states and over 
time, greatly increased the need for national training statistics. 4 

Further pressure for national training statistics capable of meeting a range of information needs 
came from various national strategies and national priorities for VET. Strategies and priorities 
relating, for example, to young people (including the 1991 Finn targets), Indigenous Australians, 
students with a disability, students in rural and remote areas, recognition of prior learning, the 
extension of apprenticeships and traineeships to existing and older workers and so on, have all 
required national data which can be disaggregated to provide statistics on a specific area of interest. 
This is one of the key reasons for having unit-record data collections rather than summary returns, 
as the latter permits no further or alternative disaggregation. 

The national collection of data on (as they are now called) Australian Apprenticeships also required 
the cooperation of both the Australian and state governments because the former is responsible for 



4 In the early years of the national data collections the raw data were often not of sufficient quality for reliable reporting 
of the key performance measures. NCVER played a key role as an adviser and data adjuster to improve the reliability of 
what was reported overseen by the National Advisory Committee on Vocational Education and Training Statistics 
(N AC VETS). 
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incentives paid to employers, while state governments have legal responsibility for the training 
contract and provide funding for the formal part of the training program. In 1997 ministers agreed 
that the national apprenticeship and traineeship statistics produced by NCVER would be the 
official statistics for reporting and accountability purposes. Previously some states had reported raw 
figures which did not take account of notification lags or administrative artefacts. 

The original driver of national data collection arrangements, namely, the information that is needed 
for monitoring and accountability, is relatively narrow, while that which is needed for reporting on 
key performance measures is somewhat broader, particularly those measures which relate to 
specific student groups. There is no question that the existence of this broader detail has had an 
‘upward spiral’ effect. Among policy developers, researchers and other users, having some 
information generates an expectation for more, on a broader scope and at a greater level of detail. 



ABS sources or new national collections? 

Grant and Fisher (1994, p.175), writing of the 1980s, note that: 

. . . little reliable assistance [on the VET sector] is gained from ABS statistics. Some of these 
latter deficiencies reflect ABS policy — leaving functional statistics to functional agencies — and 
some reflect the difficulties of collecting reliable data from such a diverse sector. 

As we have shown, the initial motivation for collections was management and accountability, 
particularly relating to the funding agreements between the Australian Government and the states. 
However, a wide range of stakeholders now expect the detailed information which we have from 
the VET collections to be available routinely. The decision of Ministers in the early 1990s to require 
complete counts of unit-record data rather than statistical returns has proved particularly far- 
sighted. Not only did it allow detailed checking — even auditing — of the data, leading to quality 
improvements, but it increased immeasurably the utility of the collections at a relatively low 
marginal cost and in ways which are not usually possible with summary returns. 

Among ABS collections, only the census supports the level of disaggregation that is possible with 
the national collections of VET provider data and Australian Apprenticeships data. In theory, the 
census could provide much of the finely nuanced detail that is needed for reporting performance 
measures, particularly those related to participation. However, the fact that this detailed information 
is only available once every lustrum, often with long delays before it is released, is a major practical 
barrier. A more serious barrier, which applies to all ABS collections and surveys, is that of reliably 
identifying VET providers and activity. 

In the community, some institutions — schools, universities and TAFE — have a clear identity. 
Beyond that, identification of VET providers by individuals is imprecise. Few would be aware, for 
example, that the Australian Red Cross is one of the country’s largest training providers, while 
many employees, including Australian Apprentices, would not be aware that training and 
assessment undertaken at their workplace is being provided by a TAFE institute or even their own 
employer if it is a registered training organisation. Also, while apprentices in traditional trades will 
be quite clear that they are doing an apprenticeship, those doing an Australian Apprenticeship in a 
non-traditional area may not identify it as anything more than paid employment. 

Similar fuzziness exists with VET qualifications. Some recognised qualifications do have a clear 
identity, such as a senior secondary certificate, a bachelor degree, PhD or a trade certificate. 
However, a person being surveyed or responding to the census might not know whether their 
certificate or diploma is a qualification under the Australian Qualifications Framework (AQF) or an 
unrecognised certificate issued by an unregistered provider. The preponderance of jargon and 
acronyms in VET is also likely to be a source of confusion among respondents and therefore of 
errors in responding to questions. 
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The problems of accurate identification are compounded when it is considered that the respondent 
to most ABS surveys, including the census, is a responsible adult on behalf of individuals in a 
household. 

While it was never practicable that the ABS could become the source of student activity data, it is 
worth reflecting on the two sample surveys instituted by the national VET system to meet its own 
requirements and collected by NCVER, rather than ABS. 5 As with the administrative collections, 
their development has been very much driven by the requirement to report progress against key 
performance measures. 

An annual survey of students exiting from the sector, the Student Outcomes Survey, has now been 
in place for over a decade. The survey is both a vehicle for monitoring students’ assessment of the 
learning experience and also of their absorption into the labour market post-training. It is with 
sample surveys that the trade-off between fine detail and the cost of collection becomes most 
pointed. Various models have been tried, including at one stage a census of all exiting students, and 
have now settled down to a sequence where each two years sufficient numbers of students are 
surveyed to provide reliable results for each TAFE institute in the country, with intervening years 
based on a much smaller sample. 

A survey of employers was first mounted in 1997. The ABS had itself conducted surveys of 
employers in relation to training practices and expenditure in 1993, 1996 and 1997 (these were 
brought together in 2001—02). The initial Survey of Employer Views began from a different 
premise. Rather than being a general survey on what employers thought of the adequacy of the 
offerings of the VET system, the initial survey was the corollary of the Student Outcomes Survey. 

It did this by, in effect, tracking the exiting students to their places of employment and asking their 
employers how satisfied they were with the skills of the student — a clear expression of who were 
the clients of the system. 

Both surveys, it might reasonably be argued, are more akin to market research than to social 
statistics, and for this reason alone ABS might well have shied away from conducting them. There 
are other reasons also. The sample of students for the Student Outcomes Survey is drawn from the 
National VET Provider Collection, which made it practically easier for NCVER to mount the 
survey in its own right. The existence of a substantial market research industry, with many players, 
some quite large, also meant that it was practically possible to collect the data independendy of ABS 

In 2005, the employer survey went through a substantial re-design to focus on employers’ use of 
the VET system in equal measure to gathering their views upon it. The sample is also now a 
general one of employers rather than those who have recently hired VET graduates. In its 
current guise, therefore, the employer survey is now much closer in purpose and sample design 
to the ABS employer survey of training practices, a survey the ABS has no plans to repeat in the 
foreseeable future. 



The importance of setting statistical standards 

The development of national VET statistical collections necessitated agreed national standards. 
Historically, VET in Australia was state-based, with each jurisdiction pursuing its own priorities and 
approaches. In TAFE institutes, student record systems, enrolment information, rules, practices 
and procedures and software systems all varied, sometimes even within a state. 

There are sound administrative reasons for all this variation and it is certainly not the job of a 
national data collection to impose some kind of uniformity — it must describe reality, not shape it. 
Agreed standards are needed to specify what information will be collected, the timing and 



5 The very first TAFE Graduate Destination Survey, the predecessor of the Student Outcomes Survey, was managed by 
ABS but all subsequent surveys have been managed by NCVER. 
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frequency of collection, classifications to be applied to describe the information that is collected 
and data formats. This is the key function of the Australian Vocational Education and Training 
Management Information Statistical Standard (AVETMISS). 

The enormous diversity of the VET system must be accommodated in a cost-effective way if the 
national data collections are to successfully meet the information needs of the sector. This diversity 
is not confined to administrative and structural characteristics. Individuals and employers use the 
system in widely varying ways. The national collection of TAFE enrolment information operating 
before 1994 assumed, in effect, that all learners in TAFE were enrolled in a course leading to a 
recognised qualification, whereas all the evidence since suggests that most learners enrol in subjects 
without completing or even intending to complete a qualification. Individuals of all ages and in all 
parts of the country access training, and the amount can vary from just a few hours in a year to 
hundreds of hours for students studying full-time at the higher AQF levels. Mobile trainers and 
training facilities are a reality, particularly in remote areas. 

Standards such as the Australian Vocational Education and Training Management Information 
Statistical Standard have an important practical function. The process of developing statistical 
standards forces stakeholders to identify and prioritise their information requirements and, because 
reliable information is never free, allows the cost and benefits to be assessed. The process of 
developing and maintaining collection arrangements and standards also promotes buy-in and 
ownership among stakeholders, an important and often under- appreciated aspect — without it, data 
collections rarely run smoothly and may even provoke considerable hostility if they are imposed 
without proper consultation and cooperative development. 

It is noteworthy that the very title of the standard reflects a dual role — to serve the management 
information requirements of providers and of state training authorities and to serve the statistical 
requirements of national consistency. This dual role of the standard is both its strength and its 
weakness. It is a strength in that the standard reflects a workable compromise between its utility to 
those at the point where the data are generated and those who are seeking to use the final 
aggregated data for statistical purposes. 

A challenge for national VET statistics has been how to accommodate the very many institutional 
changes that have taken place in the national training system. We have already noted that the learning 
programs which students undertake and the qualifications and statements of attainment they receive 
have become considerably more uniform across the country. By contrast, management, delivery and 
funding arrangements have become more diverse and fragmented, magnified by the increasing 
interest in the whole recognised training system rather than just the part that is publicly funded. 

Protocols have been developed under the governance arrangements to allow changes to the 
Australian Vocational Education and Training Management Information Statistical Standard to be 
introduced. The debates over the merits of proposed changes to the standard reflect its origin in 
serving both management information requirements and statistical purposes. In general, the 
collection of new data fields has only been agreed to where it was strictly required for monitoring 
or accountability reasons. New data fields that might serve a particular policy interest but have no 
direct utility as management information are generally not supported. An example is the proposal to 
capture information on students’ intentions at the time of enrolment, particularly whether they 
intend to complete a whole qualification or only subjects. It is also the case that, while as a principle 
the standard follows the lead of ABS in how data fields are classified, this is not adhered to where it 
may conflict with the management information requirement. An example is the classification on 
type of provider. ABS aims to distinguish between private businesses whose main activity is 
providing education and training services from other private businesses where education and 
training is a secondary activity (for example, a product supplier). The Australian Vocational 
Education and Training Management Information Statistical Standard makes no such distinction 
among private providers as, for accountability purposes, it is only the ownership of the provider 
(that is, public or private) which is of concern. 
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Embedded flaws in the design of the national 
administrative collections 



The national collections of VET data represent a major achievement and provide the sector with a 
statistical information base which few other countries can boast. However, it would be naive to 
pretend that there are not flaws and limitations, most stemming from the choices made in the early 
stages of development but some from other sources. 

A fundamental issue is defining VET without ambiguity, in a way that distinguishes it from other 
categories of learning. The first attempt at a definition might say that VET is learning that is 
intended to develop skills relevant in the workplace, including preparatory learning designed to 
articulate to such learning. But immediately there are problems because such a definition would 
include much of the learning that occurs in schools and universities. Although undesirable in a 
definition (Joseph 1906, p.98), it becomes necessary to include a sectoral restriction and exclude 
school and higher education activity. A further difficulty emerges when we attempt to establish a 
boundary between recreation, leisure and personal enrichment studies and vocational learning: 
clearly this boundary is fuzzy — is a cake-decorating course vocational or non-vocational? 

There is no theoretical way of resolving these issues: instead, pragmatism has prevailed. Recognised 
training; that is, everything which comes under the Australian Quality Training Framework 
(AQTF), is relatively easy to deal with and now constitutes the majority of VET activity. It neatly 
excludes higher education and allows VET undertaken by school students to be separated from 
their other learning. However, determining whether non-AQTF activity is VET is more 
problematic. Providers in both the TAFE and non-TAFE sectors deliver programs in areas such as 
religious, cultural and foreign language studies, and determining the status of these for collection 
purposes can be difficult. In practice, the intention of the designers of the learning program is used. 
If the intention is to develop vocational skills, then the program is VET, otherwise it is not. It is 
worth noting that the intention of learners is not used here but in reality there will be classes where 
learners have a mix of vocational and non-vocational intentions. The pragmatic resolution of the 
issue of definition has proved to be workable but remains a limitation of the collections. 

Another serious limitation is scope. Currently, the scope for the provider and financial collections is 
VET funded by state training authorities, including any fee-for-service delivery by TAFE institutes. 
However, a considerable amount of recognised VET is funded from other sources, both public and 
private, and delivered by non-TAFE providers. The consequences of not ‘knowing the full score’ 
have become of particular concern. Strategies to address this limitation are now being implemented, 
but reaching this point has taken many years of negotiation with stakeholders, particularly the 
states, which have generally shown only limited interest in the VET that they do not fund. 

Relatedly, there are major impediments to capturing management and statistical information on 
training funded by government agencies other than state training authorities. 6 

The choice of discrete, retrospective collections — administratively the easiest — in preference to a 
single national database that consolidates and updates information relating to each learner, together 
with other early decisions, has resulted in further problems and limitations. Within a single period 
there will be multiple counting of learners who undertake VET with more than one provider, 
delivery sector or state. Statistics on VET in Schools, much of which is delivered by TAFE institutes, 
are particularly affected by this issue. It is not possible to say for a given age cohort what is the total 
number of people are who involved in VET. For 1 5 to 1 9-year-olds, for example, we cannot 
separate out VET in Schools students from other VET students from apprentices and trainees. 

Information about longer-term participation and attainment in VET is also extremely difficult to 
obtain, because of the absence of unique student identifiers. Some links have since been 
established, for example, between the provider and Australian Apprenticeship collections, and 



6 Both the Australian Government and state governments are affected by this. 
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student and other key identifiers, such as course and subject codes, must now be consistent over- 
time. Even so, Australia is still some distance off a learner-centred administrative system. By way of 
comparison, the New Zealand Qualifications Authority has been operating the national Record of 
Learning system successfully for several years (NCVER 2004). This is an archive of learners’ 
achievements, with managed access for providers and learners and the capacity to issue 
qualifications and transcripts. It also captures learners’ achievements in all recognised post- 
compulsory education and training, that is, upper secondary schooling, VET and higher education. 
It is an example of a system that provides both an important service and important statistical 
information without the limitations that derive from the sectoral divisions within Australian post- 
compulsory education and training. 

Finally, various attempts to develop robust and meaningful units of measurement for VET have so 
far been unsuccessful. Counting learners — VET students and Australian Apprentices — is not 
sufficient because of the enormous variation in the type and amount of training undertaken. The 
public VET system continues to measure VET activity using annual hours — the total nominal 
hours for a year, where nominal hours are the time taken to deliver a given subject in a traditional 
delivery situation — but these measures are essentially a surrogate for funding. Annual hours have 
little comprehensible meaning outside the VET sector, and the nominal hours used for calculation 
provide an unrealistic measure when training is undertaken in a non-traditional delivery mode such 
as online or wholly in the workplace. 



Conclusion: Where to next? 

The overall conclusion to be drawn from an assessment of the development of national training 
statistics is that they are a patchwork quilt with, in places, all-too-evident seaming. The system 
works best of all in monitoring and accounting for purchasing arrangements between state training 
authorities and providers, and between the Australian Government and state governments in 
respect of funding agreements. It has also produced a time series of indicators which can be used to 
assess how well the system is performing against its objectives at a broad level. 

It is a manufactured compromise, one which tends to favour the interests of the producers of 
vocational education and training (that is, the providers and the state training authorities) over the 
consumers (that is, individual students and employers) and the general public interest. Three 
examples illustrate the point. 

First, there has been interest in some quarters in more finely grained performance indicators, right 
down to individual provider level. The rationale is that this would act as a competitive spur to 
individual providers to improve their quality, while also presenting consumers with more informed 
choices. At present, however, data protocols do not allow individual training providers to be 
identified in published reports. 

Another illustration is that the time lag between events occurring and the statistical reporting of 
those events — up to 18 months in the case of student enrolments, and around six months for 
apprentices and trainees commencing or exiting from a contract of training — is greater than some 
stakeholders would prefer. 

A final illustration is that, while the datasets have some application for secondary analysis, this is 
constrained by the inability to trace learners across collections and over time (and also by other 
factors, such as the high item non-response — far higher than would be tolerated in an ABS 
collection — on some important fields). This considerably limits the utility of the data for answering 
applied policy research questions and results in most researchers favouring the use of ABS 
confidentialised unit record files, despite the bluntness of the survey instruments. 

These three areas are all points of contention in the ongoing development of national training 
statistics and are likely to be areas where some changes may occur in the future. There are also 
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some evident areas where changes in the delivery of vocational education and training will continue 
to create challenges in the collection and reporting of national training statistics. 

The main area of a challenge lies in attempting to ensure that the collections are complete. Private 
training providers account for around a tenth of publicly funded student places and hours and also 
deliver a substantial but, as yet, unquantifiable amount of fee-for-service education and training. 
Private providers have a growing presence in both higher education and VET, delivering in niche 
areas, especially in the newly introduced qualifications of associate degrees and vocational graduate 
certificates and diplomas. To ‘know the full score’ will require extending the manufactured 
compromise (and the Australian Vocational Education and Training Management Information 
Statistical Standard) to encompass private providers. The lesson from the past is that data collected 
need to be consistent with the management information requirements of private providers. 

A second challenge, already identified above, is associated with the difficulties in identifying the 
learning pathways of individual learners, where the delivery of education and training is fragmented 
across sectors, across different types of providers within the VET sector, and across a range of 
funding arrangements (especially as the Australian Government is now a direct purchaser of some 
VET delivery). All indications are that this fragmentation will continue. This will put pressure on 
both developing cross-sectoral arrangements and information technologies for matching records 
across different administrative collections. 



References 

ABS (Australian Bureau of Statistics) & NCVER (National Centre for Vocational Education Research) 2002, 
A report to the Strategic Issues Working Group: Ret’iewofVET statistics. National Training Statistics Committee, 
Adelaide. 

Boston Consulting Group 2005, Review of implementation of peformance measures in VET, ANTA-commissioned 
report, Brisbane. 

Department of Education, Science and Training 2006, Annual national report of the Australian vocational and 
training system 2005, DEST, Canberra. 

Finn, B (chair) 1991, Youngpeople’s participation in education and training Australian Education Council, 
Melbourne. 

Grant, P & Fisher, N 1994, ‘Kangan and statistics’ in P Kearns and W Hall (eds), Kangan: 20 years on, NCVER, 
Adelaide. 

Harris, R, Rainey, L & Sumner, R 2006, Craty paving or stepping stones? Eeamingpathivays within and between 
vocational education and training and higher education, NCVER, Adelaide. 

Joseph, HWB 1906, An introduction to logic. Clarendon Press, Oxford. 

NCVER (National Centre for Vocational Education Research) 2005, ‘Recording learners’ achievements more 
effectively’. Insight, Issue 18, May. 

OECD (Organisation for Economic Co-operation and Development) 2006, Education at a glance, OECD, 

Paris. 

Wall, D 1996, ‘Strategy for transfer of functions from NATMISS Working Party to ACVETS and NCVER’, 
Advisory Committee on Vocational Education and Training Statistics Office, Canberra. 



NCVER 



31 




Discussant: Michele Bruniges, 
Department of Education and Training, ACT 



Mark and Brian touched on the topic of the patchwork nature of statistics and perhaps ‘the quilt’ 
has been stitched together from what existed and has probably been supplemented by a couple of 
additions as well. 

It was a great pleasure for me to come here this afternoon and to reply to the topic of the 
development of national training statistics. I have a real passion for data and information and how 
they can be used to make improvements in communities. In responding to Mark and Brian, I want 
to take the opportunity to pose a number of questions. I recognise that it’s far easier to ask 
questions than it is to answer them, and it’s important to reflect on what’s been achieved. 

The VET sector has come a long way along a data continuum — and a continuum of data that runs 
from ‘something that we can count’ through to ‘something that we can measure’. I draw that data 
distinction quite deliberately, and I’ll come to that a little later on. Chris Robinson’s paper noted the 
importance of research and statistics in VET policy. In 2005, the Australian Government, along 
with the states and territories, invested about $5.2 billion in vocational education and training. With 
such a large investment, we need to know how this investment has made a difference. What have 
been the costs and the benefits attributable to the investment, and indeed what is the future 
opportunity cost of not having an evidence-based approach to policy formulation? Fundamental to 
this discussion is: what are the policy questions that we are trying to answer through our collection, 
collation and dissemination of VET statistical data? 

If we look around at who’s collecting information on vocational education and training and bravely 
suggest that our operating context affords us a possibility of examining the current model, we seem 
to have lots of high-level statistics but, in terms of responding to local community needs, is the 
level of generality of the statistical information we have sufficient to respond to local needs? If we 
had the opportunity to design a national training and statistics collection, what indeed would it look 
like? How different would it be from the one that has been outlined today and what changes would 
we make, if any, and why would we do things differently? 

Any data collection, given its retrospective nature, suggests that we can predict trends using past 
trends. This can sometimes be a challenge. I don’t think we give enough thought to the changes in 
our operating context, and I do have a different view about using statistics of the past to reflect the 
trends of the future. 

We know that the Australian Bureau of Statistics (ABS) has played a role in the collection and 
dissemination of VET statistical data. I believe however that the level of detail of the data collected 
and reported is insufficient to help support key policy drivers, such as understanding skill shortages 
at a regional and local level. Data underpinning the VET sector provides an interesting model for 
education generally, and NCVER has taken a lead role in terms of providing a valuable information 
source. The school sector hasn’t had the luxury of having the equivalent of NCVER in terms of 
dealing with data, so in that context, I think the VET sector is one step ahead of the school sector. 

Data from many sources inform the VET statistical model and, in the context in which the model 
operates, the boundaries are becoming more blurred between various education sectors. So many 
organisations are involved in the collection and reporting of education and training data that I 
believe now is the time to consider more than just a national approach or a national protocol; it is 
time for us to consider a more consolidated approach to the collection and holding of data. Would 
it come in the form of an agency and protocols to use the synergies described by Mark, to look at 
individuals and learning across their lifetime? I believe that we would be better positioned as a 
nation if we carefully identified what strategic information these statistical collections contain and 
how some of the information could be more effectively utilised. It seems to me that now is the time 
for such reflection. 
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I would like to consider gaps in the data. We are collecting a huge range of data but there still seem 
to be some critical pieces missing. One of the key areas relates to staffing profile. Without that 
information, making strategic decisions about planning the workforce of the future is much more 
difficult. Furthermore, almost all of the data and information that have been referred to today focus 
on publicly funded VET. I know from recent ABS data that the publicly funded component of 
VET equates to around 65% of the total VET activity. 

What’s happening with the rest of the market? Addressing skills shortages with an incomplete 
picture means that we’re less likely to succeed as a community. Peter Kearns touched on 
globalisation and that’s a very important aspect. The changing nature of educational services 
delivery resulting from the significant increase in the use of technology, coupled with increasing 
globalisation, means that that we need to respond to the internationalisation of VET. The 
Australian VET system is widely regarded, and particularly by our Asian neighbours, as world class. 
The information we hold about the provision of Australian VET in other countries is very thin and 
leaves us further behind in understanding the overall VET market and where there are possibilities 
for expansion, and how we might address our skill needs of the future. 

It’s important not to underestimate the technological advances in relation to both the collection and 
analysis of information and how far we’ve moved down the technology path. What will be the role 
of technology in the future? How can we utilise technology further in the collection, storage, 
retrieval, analysis and dissemination of VET data? The operating context has changed and our data 
collections need to change with that context to ensure that VET remains relevant. How do we go 
about leveraging from the gains this operating context now affords us? Another question is: how 
do we improve our analysis? The funding of the VET system is currently based on inputs, but how 
do we know whether we’re getting the appropriate mix? How do we achieve a better match 
between what we need, and indeed what our workforce wants and chooses to do? 

If the VET sector is concerned with ensuring an appropriately skilled workforce, then it’s not good 
enough to fund the system purely according to the number who participate. We have to know how 
training has helped and the relevance of that training. Getting a job or the individual achieving their 
goal is part of the story, but not all. We need to understand and know how training has provided 
value. If we don’t know this, then the VET student’s experience and specific outcome may equally 
have been caused by other external factors. 

It’s very clear that we have a great wealth of information about vocational education and training 
at our finger tips. This is an outstanding achievement not realised by many other sectors. The 
data collection systems are, in the main, robust and timely. While I question the level of detail 
and the influence we have in making changes within community contexts as opposed to the 
broad national context, we need to recognise that our vocational education and training system is 
a world leader. Other countries model their training system on ours, and this brings an additional 
national responsibility. 

However, our focus is still primarily on inputs and outputs, and less on outcomes. We are 
measuring and monitoring the past very well, but how well do our current data collections translate 
to the future? What are the questions we’re trying to answer and are we blocking the continued 
growth of our system because of current structures? 

I conclude by asking how do we know that the VET system has made a difference and, even more 
importantly, what do we need to do today or what do we need to secure today to maximise the 
impact of the VET system in the future? Some of those questions have been answered this 
afternoon. When we know the answers to all of these questions, then we must sit down and have 
those hard long discussions, recognising the importance of reflection and the role that it plays in 
helping us learn and understand as a learning community. We will then be one step closer to having 
an even better VET system than we have currendy. 
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Anticipating NCVER’s 
50th anniversary? 



Tom Karmel, 

National Centre for Vocational Education Research 



It is my pleasure to round off the symposium with some views about the future of NCVER. 
Whether or not NCVER exists 25 years into the future — who knows? Twenty-five years in the 
future seems for ever but, looking back 25 years, 1981 does not seem so long ago. 

I trust that NCVER will be an enduring institution. Certainly, it has changed its name from the 
TAFE National Centre for Research and Development, and over the next twenty five years it is 
likely to change its name again, although the National Centre for Very Elderly Researchers 
(NCVER) has a certain ring to it! 

The reasons that it should be an enduring organisation are many. In my not necessarily objective 
view, its structure is ideal for a federation: owned by the nine ministers with responsibilities for 
vocational education and training, but run independently, as a company. It has large advantages 
compared with being part of a government department. First, it is at arm’s length from government 
and thus tends to be seen as neutral by stakeholders (although this perception must be jealously 
guarded). Second, its work comes out under its name and therefore does not bind its owners in 
anyway. The downside for governments is basically identical to the upside. This independence 
means that governments cannot control the release of data and research to their own advantage. 

It is a model that could be well considered for the other education sectors. 

However, being at arm’s length and having a certain degree of independence does not mean that it 
is an independent think tank. At the end of the day it is dependent on government for its finance, 
and the loss of, say, the statistics contract, would essentially doom it to extinction. Thus it must be 
very straight, and it would be very dangerous for NCVER to take a strong ideological or policy 
line. This does not mean that NCVER cannot feed into the policy debate by providing data and 
pointing to issues. Rather, it means that NCVER needs to steer carefully away from advocating 
particular positions. 

My starting point in thinking about the future is two mottos. On the bottom of our letterhead is 
. . . informing policy and practice in Australia’s training system. 

This is a very good starting point in contemplating what NCVER should be all about. I am also 
very taken by the motto of the Australian National University First, to know. By the way, this 
motto is a bit tricky and I have heard politicians leave out the comma, which gives the tag a very 
different meaning. 

My view is that NCVER’s work is all about understanding the VET sector and how it fits into our 
economy and society. The five themes we have used to classify our work give a pretty good picture 
of the scope of our work: students and individuals, teaching and learning, industry and employers, 
the VET system and VET in context. 
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In the rest of this paper, I wish to discuss in a little detail areas of work that we hope to address in 
coming years: data collections, research and analysis, and dissemination. I then want to spend a little 
time on thinking about how the environment has changed. I end with a couple of comments. 



Data collections 

This will inevitably traverse some of the same territory as Cully and Knight’s quilt, but here are 

my thoughts. 

We have currently the following collections: 

V National VET Provider Collection (students in the publicly funded VET sector, including VET 
in the adult and community education sector) 

VET in Schools 

V National Apprentice and Trainee Collection 

V National VET Financial Data Collection 

V Student Outcomes Survey 

V Survey of Employers’ Use and Views of the VET System (covering interaction with VET). 7 

While this is reasonably comprehensive and the data on the whole are of good quality, there are 

many ways the collections can be improved: 

■y- There is no staff collection — perhaps the pressures of dealing with the ageing of the VET 
workforce will be the catalyst here. 

V The apprenticeship and traineeship collection is comprehensive, but there is plenty of scope for 
it to be more useful. First, there is clearly a lot of interest in moving away from the current 
statistical concept based on the contract of training to a concept based on the individual. In fact, 
this is on our work program, and so in due course we hope to produce data on the proportion 
of people who complete an apprenticeship or traineeship. I also anticipate being able to publish 
data on completion duration (of direct relevance to current debates about accelerated 
apprenticeships), as well as being able to cut the apprenticeship data in a way better aligned to 
industry. (How many plumbers do we train for example?) An issue is a common policy 
preoccupation with total numbers rather than an understanding of the structural shifts behind 
the numbers. This was the primary motivation behind generating the ‘traditional apprenticeship’ 
series. Finally, given the interest in apprenticeships, I am surprised that we have no apprentice 
and trainee tracking survey. 

■y- Our student collection covers only publicly funded VET, and I constantiy batde to explain what 
this is in simple terms. I am hopeful that we will be able to measure all accredited VET in due 
course, including private provision; current proposals are to have a minimum data set as a 
condition of provider registration under the Australian Quality Training Framework. Even if 
we manage to crack this we will have some challenges. First, we will not be able to report on 
the total number of students (because students can attend more than one provider and we have 
no way of handling double-counting). Second, we will have to own up to the fact that our 
current collection (as does public VET funding) includes both Australian Qualifications 
Framework (AQF) accredited and non-AQF courses. This is for historical reasons, and as can 
be seen from table 1, covers higher education, secondary education, enabling courses and non- 
award courses, as well as the AQF VET courses. Thus one of the challenges will be to explain 
this to stakeholders. 



7 This is not to overlook the ABS collections such as the Surveys of Education and Training, and the annual Survey of 
Education and Work. 
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Table 1 Students by major courses and qualifications, 2000-05 



2005 (%) 


AQF qualifications 




Diploma or higher’ 3 ’ 


10.5 


Certificate IV 


10.9 


Certificate III 


26.7 


Certificate II 


15.1 


Certificate 1 


5.7 


Non- AQF qualifications 




Other recognised courses’ b) 


16.6 


Non-award courses 


5.8 


Subject only-no qualifications 


8.7 



Notes: (a) Includes diplomas, advanced diplomas, bachelor degrees, graduate certificates, graduate diplomas. 

(b) Includes statements of attainment, bridging and enabling courses, secondary education and other recognised 
courses not elsewhere classified. 



The VET in Schools data are not integrated into the main collection. This collection is in its 
infancy. The most recent collection is the first to be (mosdy) based on unit record data and I 
expect to see significant improvement in data coverage in coming years. 

There is demand for increasing the amount of data collected. For example, there is pressure for 
a ‘prisoners collection’. 

■y" A number of users have asked the question, given changes in technology, why we only collect 
and publish data once a year. 

■y- For some years we have struggled with reporting the level of activity in the VET sector. 
Flistorically, the unit of account has been ‘hours’ but this sits oddly with the focus on 
competencies, as well as being very non-intuitive to users. While ‘hours’ may have been useful 
for public accountability, they have outlived their usefulness. Another irritant has been the 
measurement of training ‘outputs’ in terms of units of competence and modules completed. Not 
only do these have no meaning for an outsider, but the two units are not commensurable (that 
is, cannot be added up). This has some import, given that one has been going up and the other 
down, and therefore it is not possible to judge whether the output of the system has been going 
up or down. I am happy to say that we are doing a considerable amount of work in this area and 
I am hopeful that coming years will see a set of more useful measures. 

Another area where we are weak is in measuring flows or pathways. While we have good data at 
a point in time, it is very difficult to measure flow concepts, such as completion rates or access 
rates (the proportion of a cohort to attend VET), or the proportion that completes a second 
qualification or articulates to university or whatever. With current data we can make estimates, 
but they tend to have heroic assumptions. To really solve this issue we need a unique identifier 
that follows students through their life. There is a lot of interest in this at the moment, but it is 
going to need a high level of political support to make it a reality. 

•y" The finance collection — or more pedantically, public finance collection — is currently coming 
under scrutiny. The impetus for this was the transfer of certain responsibilities to the 
Department of Education, Science and Training following the demise of ANTA. What is 
‘expenditure on education and training’? Does it include the payments made to employers for 
apprentice and trainees, or exemptions to payroll tax for employers? This question has 
prompted us to begin to think about the whole conceptual basis of the finance collection. 

Should we be trying to measure private expenditure? There certainly seems to be considerable 
interest from industry bodies in the contribution that employers make to training. 

To sum up, our collections form a solid base for public accountability and policy analysis but there 

is plenty of room for development. The motivation for such development is not to have better data 
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for the sake of it or for statistical ‘tidiness’ or comprehensiveness; rather, the motivation is always 
to gain a better understanding to feed into policy and practice. 



Research and analysis 

The distinction I wish to draw is between analysis of extant data (and for the sake of argument we 
can assume it is NCVER data) and research which demands a wider consideration of issues. I would 
like to spend a little time on each of these and talk about NCVER’s role. 

Taking the analysis first, I would like to provide a couple of illustrations of what I mean. The first is 
from the Apprenticeship and Traineeship Collection. 

One thing that we had observed is that the number of commencements of traditional 
apprenticeships had been increasing for some time but the number of completions had remained 
absolutely flat. 



Figure 1 Commencements and completions for traditional apprenticeships (smoothed), Australia 
1995-2005 
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Average commencements 

Average completions 
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2005 



Quarters 

Source: Apprentice and Trainee Collection 46 (NCVER) 




In a draft of one of our briefs, a colleague had written something along the lines that we expect 
the number of completions to go up in coming quarters because there was a lag between 
commencements and completions. While this was very plausible, it was not founded on any hard 
analysis. An alternative explanation was that the completion rate had dropped and we were not 
going to end up with more completions, despite the increase in commencements. To get to the 
bottom of this we set up a very simple statistical model relating completions and commencements, 
based purely on the apprenticeship data (Karmel & Virk 2007). It was an exercise that anyone 
could have done and did not rely on privileged access to the data. Anyway, the results were rather 
interesting and indicated that, yes, we could justify a statement about expecting an increase in the 
number of completions in coming quarters. 
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Figure 2 Completion rates of traditional apprenticeships, 2000-07 




■ Actual completions 

■ Predicted completions 
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Source: Derived from Apprentice and Trainee Collection 46 (NCVER) 



However, the analysis also suggested a decline in completion rates. The analysis also had an 
unexpected outcome in highlighting variations in completion patterns and rates across the states 
and territories (table 2). 



Table 2 Completion rates of traditional apprenticeships, 1998-2002, 2002-05 



States 


1998-2002 


2002-05 




New South Wales 


0.52 


0.45 




Victoria 


0.58 


0.44 




Queensland 


0.84 


0.85 




Western Australia 


0.87 


0.87 




South Australia 


0.57 


0.60 




Northern Territory 


0.79 


0.80 




Tasmania 


0.92 


0.86 




Australian Capital Territory 


0.88 


0.78 




Australia 


0.64 


0.57 





Source: Karmel and Virk (2007) 



Another example is an analysis of the divergence between student numbers and hours over a 
number of years (figure 3). 
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Figure 3 Government recurrent funded students and hours, relative to 1997 VET activity (%) 




Again there was a possible rationalisation. There has been an increase in the number of people 
undertaking higher-level qualifications and we know that higher-level qualifications tend to take 
more hours to complete. So, it could be argued, the reason for the faster increase in hours than 
students was that there had been a move to higher-level qualifications. Such an explanation needed 
scrutiny. Perhaps this was not the reason and there had been an inflation of hours instead. (Given 
the way the system is funded, there is an incentive to report increased hours.) A simple shift-share 
analysis put this to the test. By looking at the relationship between qualifications and hours of study 
and at how the profile of qualifications had changed, we can break up the increase in hours into a 
number of effects (table 3). 



Table 3 Components of change in government recurrent funded hours 





1997-2000 


2000-01 


2001-05 


Average hours effect 


-5.5 


5.2 


6.7 


Qualification share effect 


4.6 


2.3 


1.5 


Growth effect 


11.8 


-0.6 


-6.7 


Residual 


0.0 


0.0 


0.1 


Total 


10.8 


7.0 


1.5 



In this case we can see that the situation is a bit complicated and that an examination of several 
time periods is needed to understand what is going on. The bottom line is that the proposed reason 
for the divergence does not hold up. 

Three comments are in order. The first is that these types of analyses can be done by anyone; they 
are based on data that we would be happy to supply to anyone. The second comment is that the 
point of doing this type of analysis is to understand what is going on in the data. This is the type of 
work that NCVER should be heavily involved to back up its collections. The final comment is that 
this type of analysis does not explain why the data behave in the way they do. 

This last comment provides a neat link to research. NCVER is involved in research on a number of 
levels. First, it does its own research. Secondly, it synthesises research. And thirdly, it manages 
commissioned research (currently, the major program is the National Vocational Education and 
Training Research and Evaluation program). 

In the organisation’s own research I see two main types. The first is a rather more elaborate version 
of the analysis that I talked about, in which we are trying to understand better a particular topic. 

The motivation is essentially to get policy-makers to think beyond the rhetoric: sometimes to get 
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some precision on a topic, sometimes to try to explode a myth. Examples here are the work we 
have done on lower-level qualifications (Stanwick 2005a, 2005b), on higher-level VET qualifications 
(Stanwick 2006), on apprenticeships and traineeships (Cully & Curtain 2001; Ball & John 2005; 
Karmel 2006). 

To illustrate this point, I point to some work we have done on the supply of tradesmen (Karmel & 
Ong 2007). 

A common despair by commentators is that we are running out of tradesmen. Not enough young 
people enter them (university is too attractive), and current tradesmen are middle-aged and about to 
retire. The ageing of the population means that we can no longer rely on young men to maintain 
the supply. But do these views really stack up? To answer this, we built a simple supply model in 
which people enter apprenticeships, either complete or not complete them, and then leave the 
occupation at some point. The data used comprise NCVER apprenticeship data on 
commencements and cancellations and withdrawal, and ABS occupational employment data. By 
making assumptions about commencement rates, cancellation rates, and attrition rates, all based on 
what we have observed historically, we can project forward the supply using ABS population 
projections. We do this by making use of a simple accounting formula that says that the number of 
employed persons in a trade equals the number last year, plus new apprentices, less apprentice 
withdrawals, less net attrition. These projections can then be compared with demand projections 
based on the output of a model such as Monash. What we learn from this exercise is that much of 
the current rhetoric is debatable. Unless the demand projections are completely wrong, there is not 
much of an issue from the demographic angle. Moreover, in most trades there will not be much of a 
change in the age profile. 

The other type of research tries to get behind the numbers. An example of this is the work 
currently being completed by Misko and colleagues (2007), in which we conducted a survey and 
focus group discussion on attititudes to apprenticeships of students in Years 10 to 12 and with 
apprentices themselves. 

Looking forward, there will always be a demand for this type of research. 

The demand for the next category, synthesis, seems to be largely insatiable. Synthesis takes a 
number of forms: short publications (At a glances) that bring together a range of material on a topic, 
and more sophisticated surveys (Systematic reviews). The latter has been a recent development, and 
my judgement is that they are a lot of work, but potentially quite rewarding. We have now done 
three: education and training and older people, what works for Indigenous Australians, and what 
works for small business. The main challenge to the systematic review is that they can expose real 
weaknesses in the research, and it always sits uncomfortably to spend large resources to conclude 
we need more research. 

Of the three I think the Indigenous review has had the best pay-off. The evidence here was pretty 
definitive and the conclusions (box 1) are clear. 

Box 1 Findings from the systematic review into Indigenous Australians and VET 



Through a systematic review of existing research, clear evidence has been found that seven key factors lead 
to positive and improved outcomes for Indigenous Australians as a result of vocational education and 
training. These are: 

community ownership and involvement 
A the incorporation of Indigenous identities, cultures, knowledge and values 
<r the establishment of ‘true’ partnerships 

flexibility in course design, content and delivery 
A quality staff and committed advocacy 
<r extensive student support services 
A appropriate funding that allows for sustainability. 

Source: Miller (2005) 
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In terms of the future, one area I would support would be an in-depth survey of recent research on 
VET teaching and learning. There is a real debate about what goes on at schools at the moment 
(outcomes versus getting back to the basics), but I don’t see a corresponding policy debate in VET. 

In terms of the research that we manage, there are a number of issues that I would like to point to. 

First, it is timely to re-examine the relative efficiency of funding individual projects. The consortium 
approach in which a group of researchers is funded at very significant levels to look at a particular 
topic (see box 2 for the two currently being funded), and the research centre approach in which 
funding is attached to a centre rather than a project might be more efficient models. 



Box 2 The consortium approach to funding individual projects 



Consortium 1 

The program, ‘A well-skilled future: Tailoring vocational education and training to the emerging labour market’ 
was undertaken by a consortium of researchers from Flinders University’s National Institute of Labour 
Studies (NILS) and Melbourne University’s Centre for Post-compulsory Education and Lifelong Learning. 
Comprising a suite of 13 reports, the research program investigated the nature of future labour demand and 
supply, the changing nature of work organisation, the participation of existing and new workers in VET, and 
the VET sector’s current and future capacity for change and adaptation. 

Consortium 2 

The research consortium, ‘Supporting vocational education and training providers in building capacity for the 
future’, comprises researchers from the Centre for Research in Education, Equity and Work at the University 
of South Australia, the Centre Undertaking Research in Vocational Education at the Canberra Institute of 
Technology, and the Australian Centre for Organisational, Vocational and Adult Learning at the University 
of Technology, Sydney. Comprising nine reports, this consortium investigated how VET organisations are 
building capability for and dealing with an increasing complex and rapidly changing environment. 



The second issue is how to build researcher capacity, noting that the research depth is not 
particularly satisfactory in VET research. We are currently looking at a range of possibilities, 
including research scholarships and mentoring grants, and the 2007 NCVER ‘no frills’ conference 
has building capacity as one of its themes. 

The third issue is research quality. While it is always easy to be critical, we continually struggle to 
keep researchers on the straight and narrow: a logical line of argument or story, and clear, simple 
and succinct expression. The way we manage our commissioned research work is quite different 
from the general approach. We put in a huge effort to get work to what we believe is an appropriate 
standard. By contrast, the more usual approach is to let researchers sink or swim. If the work is not 
up to scratch, it will not be published and future grants will not be forthcoming. However, this is 
not appropriate for VET research because of its very applied nature. If the work is not 
disseminated to the policy community and practitioners, then it has not achieved its purpose. There 
is little point in waiting for research to be published in the Annals of A r ET research or whatever; I 
suspect that our target audience does not spend too much time reading scholarly journals. 



Dissemination 

The last comment provides a neat transition to something that is very important to NCVER. We 
currently put a huge effort into dissemination and I expect that it is something which will dominate 
the next 25 years as well. Dissemination is about product and method. In terms of our research the 
major innovation in recent times has been the 1:3:25 research report, now transmuted into the 
1:1:3:25 (box 3) research report. This seems to have been a relative success with readers, although 
some researchers feel constrained and, to examine their concerns, we plan to review the format. In 
implementing this format we are now contemplating not asking authors to write the key messages. 
It seems that often they are too close to the work and struggle with distilling the three or four key 
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messages that the journalists or senior policy-makers really want. Another product that seems to 
work well is the one-page flyer. 

Box 3 Structure of NCVER research reports 



1 page: the foreword, to provide context 

1 page: key messages, to provide the three or four key points for policy-makers or journalists 
3 pages: executive summary, a self-contained essay on the report 
25 pages: the body of the report 
Unlimited pages: supporting documentation 



However, written reports are only one thing we do. We also organise research forums and road 
shows, and do a very large number of presentations. One format that works very well is a 
presentation to a relatively small number of people from an organisation, allowing interaction 
and discussion. 

As well as these rather conventional approaches we are thinking about other possibilities. There is 
some interest in pod casts and chat rooms but I will leave it to others to argue their benefits. 

I would like now to move to statistics. This is an area that is changing fundamentally because of the 
ability to present huge amounts of data on the web for a low cost. The days of the hard copy 
statistical publications are numbered, and we are currently putting quite an effort into moving away 
from them. The direction of the future is fairly obvious. First, we will be able to publish a huge range 
of tables, generated automatically (not hand-crafted) and unconstrained by space limitations. The 
great benefit of this is that we can cater for very specific interests at a low cost. For example, there is 
no reason why we could not publish a series of plumber apprentice numbers. We publish additional 
tables on the web at the moment to some extent (for example, state tables for Students and courses are 
published on the web but not in hard copy) and this is an area I envisage will grow exponentially. 

The other type of product will have the capacity to enable the user to construct their own tabulations 
interactively. We are working on this at the moment using Space-time research technology. The idea is 
that users can download the reading technology for free and then use it to build their own tables. The 
only constraints then will be the statistical classifications built into the collections, 
privacy/ confidentiality considerations and, in the case of survey data, sampling error considerations. 

In the same vein there is no reason why we can’t have different ‘cuts’ of the data. The obvious 
example, and one we are working on, is the apprenticeship and traineeship data in which it is 
possible to have a contract of training view (the current approach) or a view based on individuals 
(allowing individuals to change their employers and occupation). 

Another area where there are potentially important changes is in making research material more 
readily available. We are constantly looking at how VOCED, our research bibliographic database, 
can be made more effective. At the moment we are investigating how we can link it to other 
research databases such as ERIC, and whether we could maintain a digital library of all the materials 
referenced in VOCED. NCVER could act as an online library to the sector. 



Some final comments 

I would like to end with a question and a comment. First, I wish to ask the question whether 
there are potential areas that NCVER should not get into. Second, I wish to think a litde about 
the policy environment. 

There are two areas that I think would be problematic for NCVER. The first is that of policy. My 
view is that NCVER needs to be very careful about getting too close to policy. It is absolutely 
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critical that we feed into the policy debate through the provision of information and research, and I 
have no objection to putting conventional wisdom to the test, but NCVER is not a ‘think tank’ and 
should not be pushing a line. The second is policy evaluation. My feelings here are a little mixed. I 
certainly have no objection to NCVER undertaking an evaluation of a government program on a 
consultancy basis, in which case the report is clearly the property of the client. However, it would 
not be appropriate for NCVER to do its own evaluation of a particular government program. This 
is not to say that we cannot look at issues that relate to government programs. For example, in 
recent times we have seen the introduction of the Australian Technical Colleges and state trade 
schools. It will be up to the Australian Government to evaluate the technical colleges, not NCVER. 
However, this is not to say that we can’t do work in the area, and indeed we plan to produce 
apprenticeship and traineeship data that compares the performance of school-based apprentices 
and trainees with that of their non-school peers. 

To look forward it is sometimes useful to look backwards. I worked at the Bureau of Labour 
Market Research in the mid-1980s. This organisation had some similarities with the NCVER in 
terms of structure (independent from a government department) and subject matter. Clearly, 
NCVER has done rather better, and in fact the Bureau of Labour Market Research had a relatively 
short life. This I believe was primarily because such organisations do not sit comfortably within 
government in Australia. NCVER’s structure, being owned by nine governments that typically 
cover both the Coalition and Labor dimensions of party politics, is a much sounder foundation. 

My final comment relates to the economic environment. If you look at the issues back then and 
now, they are remarkably similar. Labour markets remain labour markets. There is still an emphasis 
on understanding how labour markets work (and vocational education and training is essentially 
about labour markets) and how various groups are faring. What is different is that in the 1980s the 
issues were driven by unemployment; in 2007 the drivers are skill shortages and how to improve 
the quality and quantity of labour supply. Structural adjustment was a big issue then, less so today. 
However, the drought and global warming may well force big adjustments in the economy that 
would change the focus yet again. The point of this comment is that the issues that drive our work 
will not go away and, from this perspective, NCVER’s future is assured. We will never get to the 
point where there is nothing more to learn. There will always be a need to provide information and 
research to aid the understanding that policy-makers and practitioners need. 
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